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Related Application 

This application claims the benefit of U.S. Provisional Application No. 
60/255,541, filed December 14, 2000, incorporated by reference herein. 
Background of the Invention 

Some interactive programs include digital video discs (DVDs) or resident client 
computer applications, which enable users to view ancillary content that relates to the 
primary content being viewed. For example, DVD movies (the primary content) can 
include other footage relating to the movie, such as interviews with the cast, the movie 
trailer, and outtakes. DVDs also exist that permit the user to connect to a universal 
resource locator (URL) through a browser when viewed on a personal computer. This 
experience, however, is limited in that the user must use the DVD to make such a 
connection and the user is unable to pause the video to interact with any ancillary 
content. In the case of DVDs without such Internet connectivity, the user can only 
navigate through the ancillary content embedded in the DVD, and cannot perform any 
real-time transactions (e.g., buy a copy of a movie soundtrack through an e-commerce 
transaction) or other user interaction. 

Other interactive programs permit users to watch content (e.g., a television 
episode) and log onto a website afterwards to purchase items viewed during the show 
(e.g., a wristwatch worn by an actor in the show). 

Some interactive programs in a broadcast environment utilize the vertical 
blanking interval (VBI) to insert data into the broadcast stream, thus enabling Interactive 
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functionality. For exannple, with web television, users may play along with game shows 
as they are being broadcast. However, this approach is limited to the broadcast arena 
where the primary content (e.g., a television show) cannot be interrupted while the user 
interacts with ancillary content. 

While the foregoing interactive programs provide users with an enhanced 
experience, they are limited in providing real-time interactivity between the user and the 
content while the user is viewing the primary content and do not provide a user-friendly 
experience. Therefore there exists a need for interactive video content programming 
that permits the user to stop the video play to view ancillary content, and then continue 
video play from the point in time where play was stopped. 

SUMMARY OF THE INVENTION 

The present invention is directed to a system and methods for creating and 
distributing interactive video content (IVC). IVC includes the creation of interactive 
content using software tools (e.g., Flash™ and Shockwave®), and digital assets (e.g., a 
movie or television commercial), and distributing the created interactive content in real- 
time to a user over an Internet Protocol (IP)-based network (e.g., Internet and intranet), 
or other network supporting two-way communication, to provide an interactive user 
experience. 

As used herein, the term "content" is meant to Include all forms of viewable 
electronic information including, but not limited to, advertisements, promotions, music 
videos, motion pictures, and television programs. A preferred embodiment of the 
present invention is directed to a method for using an interactive video including 
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displaying a video on a visual display, the video having at least one interface link 
associated therewith, the interface link adapted to be displayed on the visual display 
and being linked to ancillary content accessible over a network (wire or wireless); 
interacting with the interface link to access the ancillary content; interrupting the display 
of the video at a point in time; delivering the ancillary content to the visual display; and 
continuing the display of the video from the point in time where the display of the video 
was interrupted. 

As used herein, the phrase "ancillary content" is meant to include any content or 
page of content linked to the primary content or content linked therefrom. Also as used 
herein, the phrase "visual display" is meant to include all types of video or audio-visual 
devices including, but not limited to, screens for computer and televisions, personal 
digital assistants, or any other device that provides visual content to a user. As used 
herein, the phrase "interface link" is meant to include any means that functions as a link 
between video content and another piece of content, for example, a hypertext link 
under an Internet protocol regime. 

Each interface link is preferably associated with, or related to, content being 
displayed on the screen. For example, if the user is watching a basketball game, and 
the user is interested in a particular shoe worn by a basketball player, the user may 
select the interface link associated with the basketball player's shoe. Interacting with 
the interface link associated with the basketball shoe allows the user to access one or 
more pages of information or media content related to the shoe of interest, including 
retail information. During the user's interaction with the interface link, the video stream 
is paused until the user returns to or continues the video stream delivery. Thus, a user 
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may freely interact with one or more interface links to gain more information about an 
object of interest being displayed without missing any of the primary content video. As 
used herein, the phrase "primary content" is meant to include any content first 
requested by or to be shown to the user. 

5 The present invention is also directed to a method for creating an interactive 

video, including creating a link program adapted to interrupt the delivery of video to a 
visual display and provide access to ancillary content accessible over a network; 
encoding the video onto a storage medium adapted to store video content; associating 
the link program with the video; delivering the video to the visual display; and displaying 

iffj the video on the visual display. 

jfli Once an interactive video has been created, it may be distributed in several 

J!^: ways, A preferred distribution channel is to stream the video over an Internet Protocol 
;5 (IP)-based network (e.g., Internet and intranet). Interface links may be displayed with 
r| the video stream in several ways. For example, interface links may be delivered 
ly separately from the video stream such that the links overlay the video stream content 
O when displayed to the user (a "floating" interface link), or the interface links may be 
embedded in the video stream itself. Delivering interface links separately from the 
video stream eliminates any need to modify the original video content to support one or 
more interi'ace links. Interaction with the interface link provides the user access to at 
20 least one IP address, for example, a web page address. 

The present invention provides real-time interactivity that permits the user to 
effortlessly make a real-time transaction during the viewing of the program. The 
present invention may also be used for advertisements and specialized e-commerce 
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opportunities. 

It is to be understood that both the foregoing general description and the 
following detailed description are exemplary and explanatory only and are not restrictive 
of the invention, as claimed. 

The accompanying drawings, which are incorporated in and constitute a part of 
this specification, illustrate several embodiments of the invention and together with the 
description, serve to explain the principles of the invention. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Fig. 1 is a schematic diagram of system components in accordance with a 
preferred embodiment of the present invention; 

Fig. 2 is a schematic diagram of system components in accordance with another 
preferred embodiment of the invention; 

Fig. 3 is a logic diagram of a preferred method for creating interactive video 
content; 

Fig. 4 is logic diagram of a preferred method for delivering interactive video 
content; and 

Fig. 5 is a logic diagram of a preferred method for conducting e-commerce 
through an interactive video. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 

Reference will now be made in detail to the present preferred embodiments 
(exemplary embodiments) of the invention, examples of which are illustrated in the 
accompanying drawings. 

The present invention is directed to a system and methods for creating and 
distributing interactive video content. Unless otherwise stated, the present invention will 
be described in relation to using streamed video over an IP-based network such as the 
Internet, although a person of ordinary skill in the art will appreciate that other means of 
video delivery are possible and within the scope of the present invention. Fig. 1 is a 
schematic diagram of a preferred embodiment of the invention. As shown in Fig. 1, 
endpoint servers 100 deliver media content (e.g., movies, television shows, and ads) 
through a network 102 to one or more users 104. Network 102 may be any network 
that accommodates electronic delivery of content including, but not limited to, satellite 
transmission, wireless networks, digital subscriber lines, cable, and other 
communication networks. Endpoint servers 1 00 are preferably located in the vicinity of 
a service provider, such as an Internet service provider for example, in order to utilize 
the service provider's broadband network for streaming media content to a user and 
allow for more efficient communication between user 104 and endpoint server 100. 

Endpoint servers 100 preferably include a video server 106, a web server 108, 
and a content database 110. It should be understood that endpoint servers 100 may 
include only one server. Video server 1 06 may be any server adapted to store and 
provide access to video content suitable for streaming to users. Web server 108 may 
be any server adapted to serve static images (e.g., JPEG or GIF), HTML assets (e.g., a 
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retail website), text, and other IP-based file types (e.g., Flash™ and Shockwave®). A 
preferred form of web server 108 is a HTML server. Content database 110 preferably 
stores data for use with web server 108, and metadata associated with video content 
stored on video server 106 and may have a storage capacity expandable by known 
methods. It will be appreciated by those of ordinary skill in the art that in any of the 
embodiments of the present invention, the number of servers may range from one to 
many depending upon the system requirements to be met. Likewise, the system 
architecture between individual servers may be varied and load-balanced in known 
ways in order to provide optimal system efficiency. 

Fig. 2 shows a preferred embodiment of the system of the present invention with 
the system components in a centralized architecture. Central servers 200 include a 
web server 208 and a content database 210. Web server 208 and content database 
210 have similar functions as web server 108 and content database 110, respectively, 
shown and described in Fig. 1 . Instead of being located regionally or locally in the 
vicinity of an Internet service provider, web server 208 and content database 210 are 
centrally located to provide access to a plurality of users 204 over a plurality of local or 
regional networks 202, each network 202 having associated therewith at least one 
video server 206. By centrally locating web server 208 and content database 210, 
media content may be more efficiently managed, for example, by reducing the amount 
of duplication otherwise required with greater numbers of servers and databases. It 
should be understood that central servers 200 may include only one server. 

Fig. 3 shows a preferred method for creating interactive video content. In step 
300, a preferably non-linear storyboard is created. A non-linear storyboard sets forth a 
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flow of screens and/or images to be presented to the user during the user's interaction 
with the video content, A user may access one or more storyboards during the 
streaming of the video content via an interface link. Each interface link is preferably 
associated with a storyboard. Selecting or otherwise interacting with an interface link 
permits a user to access information or media content associated with a streamed 
object that the interface link is linked to. The storyboard preferably includes a "hub 
page" (the first page the user sees after interacting with the interface link) and at least 
one "sub-page" (any page linked to the hub page). Any of the storyboard pages may 
be, for example, a web page using graphics and/or text, or a streamed video asset. By 
being non-linear, a user may be presented with a plurality of choices at each level of 
selection (e.g., at the hub page and any number of subsequent sub-pages). 

In step 310, video and/or audio assets are encoded. Encoding step 310 includes 
converting video and/or audio assets into computer files that are readable by an 
application adapted to show video to a user, for example, a media player application. 
Video content is preferably encoded as ASF, MPEG4 files to take advantage of 
encryption opportunities. However, interactive media files may also be encoded as, for 
example, Quick-time™ files or AVI video files. A preferred encoding software is 
Windows® Media Encoder 7.0™. Preferably, both primary video content (i.e., the video 
initially requested by the user) and any ancillary video content (i.e., video that may be 
viewed while the primary video content is paused) are encoded onto the same storage 
medium. As a person of ordinary skill in the art will recognize, various software 
applications may be used to encode content without departing from the scope of the 
present invention. 
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In step 320, one or more hub pages may be created. Hub pages are created 
using any software tool adapted to create and populate IP-based pages (e.g., web 
pages). Preferred software includes, for example, Flash™, Shockwave®, HTML, and 
DHTML. In step 330, one or more sub-pages are created using tools such as those 
used to create hub pages. Hub pages preferably include a link back to the point at 
which the video content stream was paused or interrupted, and one or more links to 
sub-pages. Sub-pages themselves may include links to more sub-pages, or a back link 
to the hub page or to the interruption point of the video. Each hub page or sub-page 
may include one or more links to commerce sites. As used herein, the phrase 
"commerce site" is meant to include a site residing at an electronic address that is 
adapted to handle commercial transactions, for example only, a retailer website using 
an IP address. 

After the hub pages and sub pages are created, they are preferably entered into 
a content management system for tracking and display purposes. "Tracking" includes 
accounting for the exhibition of the video asset by means of a unique identifier. A 
preferred example of a content management system operable with the present 
invention is taught in U.S. Application Serial No. (to be assigned), titled "Content 
Management System," filed July 31 , 2001 , which claims priority to U.S. Application No. 
60/280,691 , the disclosures of which are hereby incorporated by reference herein. 

The creation of links is described in more detail below. Hub pages and sub- 
pages may be stored, for example, on web server 108, 208, a retailer site, or on client 
software at the user's location. 
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In steps 340 and 350, interface links are programmed according to tlie intended 
metliod of presentation and associated with a piece of video content, whether primary 
or ancillary. 

Interface links may be presented in several ways on a user's visual display. For 
example, interface links may be embedded in the video content such that the links are 
streamed with the video content from video server 106, 206. Embedded interface links 
may be created by on-line software such as Smoke® (available from Discreet Logic™), 
Final Cut Pro® (available from Apple Computer™), or Avid® (available from Avid 
Technology™). Preferably, the embedded interface link is located in the lower left hand 
corner, inside the video delivery area intended for the visual display, just outside the 
intended delivery area for Internet protocol. 

Interface links may also be hidden from view such that no icons are visible. In 
this instance a user may, for example, when selecting with a mouse, run the cursor over 
an object of interest (an on-screen object that the viewer is interested in learning 
additional information about). When the cursor contacts the on-screen object (e.g., a 
shoe worn by a player during a basketball game) an icon or other visual effect may 
appear signifying that an interface link is available for the object of interest. Hidden 
interface links may be created by embedding an interface link as an invisible layer on 
top of the streaming video with known editing applications that can generate 
navigational instructions via, for example. Lingo™ (available from Macromedia®), 
Visual Basic® ("VB"; available from Microsoft®), ActiveX® (available from Microsoft®), 
Coms, or DirectX® (available from Microsoft®). 
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Hidden interface links may be placed on the canvass of a video streann over a 
single pixel, or over a greater number of pixels about tlie display area. Hidden interface 
links may be adapted to serve a number of purposes sucii as detecting a full screen 
event like an indiscriminate key stroke or mouse function to trigger an event such as an 
HTML page call or a chapter advance to another video sequence. A hidden interface 
link may also be used with a single pixel to make a cursor change indicate a hot mouse 
event on a specific part of the video stream, or be used for marketing and/or user 
feedback. 

Interface links may also be delivered from web server 108, 208 and shown on 
the user's display as an overlay to the streaming video, for example, as a visible, 
translucent icon (e.g., "floating bug") or other user interface (Ul). User 104, 204 would 
then be receiving two simultaneous transmissions: one from video server 106, 206 and 
one from web server 108, 208. Interface links delivered from web server 106, 206 may 
be delivered as a timed program that coincides with the video content being streamed. 
In such an instance, interface links may be preprogrammed to interact with, for 
example, time code markers embedded in the video stream, such that one or more 
interface links may appear or disappear based on the time elapsed. The association of 
interface links with time code markers may be achieved by known video editing or 
encoding applications. The appearance of a time code marker may be triggered when 
a time code window of the application delivering the video, for example, a media player, 
reaches a selected frame. For example, an interface link may appear in the right hand 
corner of the user's display after five minutes have elapsed during a video presentation 
to coincide with the entrance of an object of interest (e.g., an automobile coming from 
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the right corner of the display). This process is akin to laying a template over the user's 
display, rather than embedding interface links in the video stream. 

It is appreciated that an interface link program may be delivered to client 
software operatively connected to the user's visual display to interact with video 
delivered from video server 106, 206. In this embodiment, delivery of the interface link 
program need not be simultaneously delivered with the video to the user since the 
interface link program would already be at the user's visual display. Links to and 
between non-video content between pages may also be programmed as needed. 
Multiple links may be associated with the video using a variety of formats (e.g., hidden 
or translucent icons) which change with both time and location as the video plays. 

In step 360, video content is distributed to one or more video servers 106, 206. 
Video may be distributed by any means adapted to deliver video content from one 
location to another, for example, manual delivery, satellite transmission, wireless 
delivery, digital subscriber line, and cable. In step 370, hub pages and sub-pages are 
distributed to web server 108, 208. In step 380, metadata is distributed to content 
database 110, 210. Distribution mediums may be the same as those already 
mentioned in conjunction with video content distribution. It should be understood that 
the aforementioned steps need not be performed in a particular order. For example, 
the video assets may be encoded before the creation of the storyboard in step 300. 
The creation and distribution of the hub and sub-pages to web server 108, 208 may 
occur independently of the creation and distribution of the video assets to video server 
106, 206. In addition, the creation and distribution of metadata to content database 
110,210 may occur independently of either of the above. 
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Fig. 4 shows a preferred method for delivering interactive video. In step 400, 
video servers 106, 206 receive a video request. In step 402, video servers 106, 206 
stream requested video to the user's visual display. A preferred delivery program for 
streaming video content to user 104, 204, is Windows Media Player™. During video 
streaming, one or more interface links are delivered to the user's display and 
associated with the displayed video. 

As a user is receiving a requested video stream, the user may be presented with 
one or more interface links. If the user decides to interact with an interface link In step 
404, then the user selects a desired interface link corresponding to an object of interest. 
A user may interact with an interface link by, for example, touching an area of the 
display, voicing a command, pointing and clicking with a mouse, using a beam of light 
aimed at an area of the display, or any other interaction that conveys the user's desire 
to interact with an interface link. Once an interaction with the interface link has been 
detected, in step 408 the video stream is paused or interrupted. 

In step 410, the IP address associated with the interacted interface link is 
accessed. For example, in a web setting, a web page address is accessed and the 
user request is sent to the URL for the hub page, which is preferably served from a 
centrally located HTML server. In step 412, a hub page and any associated metadata 
with the accessed address are delivered to the user. The user may then view the hub 
page and make a sub-page selection in step 414. If the user selects a sub-page, then 
in step 416 a selected sub-page and any associated metadata Is delivered to the user. 

In step 418, the user may decide whether to request a different hub page, a 
previous hub page, or a new sub-page (though not illustrated, step 418 may loop to 
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step 416 for as many times as a user desires to access a different sub-page). If the 
user decides not to select any furtlier pages, then in step 420 the user may decide 
whether to continue the video stream in step 422. If the user does not elect to continue 
the video stream in step 420, then the user may continue to view the page that the user 
is viewing, or select one or more new pages and continue the video stream at anytime. 
Choosing to continue or return to the video stream will bring the user back to the point 
where the video streaming was interrupted. This may be done by a user action which 
activates the browser window containing the video stream. 

After the video streaming has been continued, the user may select another link 
and thus repeat steps 404-422. It should be understood that the aforementioned steps 
need not occur in a particular order, or include all steps. For example, hub pages are 
not required to have sub-pages associated therewith. Therefore, in instances where a 
hub page has no associated sub-page, steps 414-418 may be omitted. 

Each hub page or sub-page may contain e-commerce opportunities, i.e., retail 
information and/or links to retail sites for ordering desired items and completing 
commercial transactions. For example, during the presentation of an action film 
showing a snow-ski chase, a user might be interested in the brand of skis that a 
particular actor may be wearing. The user may select an interface link associated with 
the ski shown on the user's display. If the user is using a computer with a mouse, the 
user may simply point and click on the ski of interest, thereby pausing or interrupting the 
movie and delivering a hub page showing retail information regarding the particular ski 
of interest. The user may then choose among different sub-pages showing more 
information about the desired ski, or may order the ski from one or more of the pages. 
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Sub-pages may themselves contain video assets. For example, if a hub page 
contains information about an automobile, a sub-page \mk might lead to a video 
demonstration of the automobile's performance in various conditions. After exploring 
the hub page and any of various sub-pages, the user may elect to return to the primary 
video content at the point of interruption. It will be appreciated that the hub pages may 
or may not include ancillary video assets depending upon, for example, the system 
requirements and the storyboard intended to be designed around the primary content. 

Interface link graphical Images or icons (for visible icons) are preferably 
translucent to provide little distraction to the user during the video content presentation. 
For example, a preferred interface link includes a graphic that is slightly beveled, fifty 
percent transparent, and approximately 60 by 50 pixels in size. The icons may be 
created software such as, for example, Adobe® Photoshop™ and others. 

Fig. 5 shows an example of an interaction with a storyboard having e-commerce 
opportunities. The user first requests the primary content (e.g., movie). When such a 
request is made, the primary content is streamed from video server network 106, 206 to 
the user with a marker indicating the availability of interactivity (e.g., the "floating bug"). 
If the user is interested in obtaining information about an object being displayed in the 
video (e.g., a coat worn by an actress), then in step 500 the user selects the "floating 
bug" associated with the object of interest. The primary content video stream is paused 
while the user request for information is sent to the URL for the hub page located on 
web server 108, 208. The hub page is then served to the user from web server 108, 
208. If the user is interested in purchasing the object of interest (e.g., the coat), or any 
other object or service contained on the hub page, then in step 502 the user selects an 
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interface link 503 labeled "buy now" associated with the object or service the user is 
interested in purchasing. Selecting Interface link 503 connects the user with a retail 
web site having purchasing opportunities for the objects or services of interest. In step 
504, the user proceeds to buy merchandise from the retail site. The retail site may be 
operated by a commercial entity responsible for presenting the primary video, or an 
outside entity. The retail web site may be a sub page served by web server 108, 208, 
or a web site located outside the system. At any point, the user may select an icon to 
revert to a previous page or the primary content. Set forth below are examples of 
preferred embodiments of the system and methods of the present invention: 
Interactive Content Programming (ICP) includes several features: 

1 . ICP-enabled content will be visually distinguishable from other content via the 
translucent "bug" or other user interface (Ul) linking element floating over the video 
content. The "floating bug" provides an interactive experience without modifying the 
primary content. This Ul element both signals ICP availability, and accepts user "clicks" 
to trigger transition to the linked content. 

2. If the ICP linking element is clicked, the Ul Is redirected to a programmed "place" 
(e.g., web page) that may include a variety of Interactive content options. The place will 
be specified as a URL to be loaded over a current frame, or in place of the current 
page. The place may be an e-commerce opportunity, another video segment, or the 
like. In the most general case, any arbitrary URL (in any web-friendly format) may be 
the target. Preferably, multiple linkages from the video will be offered in a variety of 
formats (e.g., other than the translucent "bug"), which may be adapted to change with 
both time and space as the video plays. 
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3. When the user is done with the linked content and returns to the original 
video, it resumes at the point at which the user left it (i.e., the user does not miss a 
frame of original video). 

An example of creating an ICP includes the following steps: 

1 . Create a non-linear storyboard. An example of a non-linear storyboard is 
found above in Fig. 5. The storyboard typically Includes a "hub" page which is the first 
page the user sees when they click the "bug" to access the interactive content. The 
"bug" can link to the hub page or any page linked to from the hub. The hub page and 
all other sub pages preferably include a link back to the video where it left off, or in the 
case of a sub page, a link back to the hub page. 

2. Encode the video assets and/or audio assets (e.g., the primary content and 
any ancillary content used on sub and hub pages) using encoding software (such as 
Windows Media Encoder 7.0™). 

3. Utilizing software tools (e.g., Flash™, Shockwave®, HTML), create the hub 
page (e.g., a web page using graphics, text). 

4. Utilizing software tools (e.g.. Flash Shockwave®, HTML), create one or 
more sub pages (e.g., a web page using graphics, text). 

An example of deploying an ICP includes the following steps: 

1 . For deployment of an ICP, the content management system allows content to 
Include a URL for linking and accommodates "floating bug" insertion. 

For each video asset associated with the ICP, the hub pages and sub pages are 
entered into the content management system for tracking and display purposes. 

2. All HTML assets are placed on a central HTML server, which can be 
accessed by multiple versions of client applications (e.g., ".com" and ".tv" entities). 
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3, All of the video assets are distributed into the video server network (e.g., 
Akamai™). The video server network distributes the video content nationwide to 
individual users (i.e., client applications). 

4. The client application is enabled to exhibit the ICP by modifying the content 
database accessible by the public to include the ICP. 

Now the user can select the ICP and receive an interactive experience by 
requesting the primary content (e.g., movie). When such a request is made, the 
primary content is streamed from the video server network to the client application with 
the "floating bug." Upon the user selecting the bug (e.g., by clicking on the "bug"), the 
primary content video stream is paused and the user request is sent to the URL for the 
hub page located on the central HTML server. The hub page is then served to the 
client application from that central HTML server. The hub page may or may not include 
ancillary video assets also served from the video server network. A user may explore 
the hub pages and sub pages including any ancillary video assets and at any point click 
to return to the primary content where they left. 

Other embodiments of the invention will be apparent to those skilled in the art 
from consideration of the specification and practice of the invention disclosed herein. It 
is intended that the specification and examples be considered as exemplary only, with a 
true scope and spirit of the invention being indicated by the following claims. 
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